Unified Pragmatic Models for Generating and Following Instructions

We extend models for both following and generating natural language instructions by adding an explicit pragmatic layer. These pragmatics-enabled models explicitly reason about why speakers produce certain instructions, and about how listeners will react upon hearing them. Given learned base listener and speaker models, we build a pragmatic listener that uses the base speaker to reason counterfactually about alternative action descriptions, and a pragmatic speaker that uses the base listener to simulate the interpretation of candidate instruction sequences. Evaluation of language generation and interpretation in the SAIL navigation and SCONE instruction following datasets shows that the pragmatic inference procedure improves state-of-the-art listener models (at correctly interpreting human instructions) and speaker models (at producing instructions correctly interpretable by humans) in diverse settings.